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Abstract 

Random sampling in compressive sensing (CS) enables the compression of large amounts of 
input signals in an efficient manner, which is useful for many applications. CS reconstructs the 
compressed signals exactly with overwhelming probability when incoming data can be sparsely 
represented with a few components. However, the theory of CS framework including random 
sampling has been focused on exact recovery of signal; impreciseness in signal recovery has 
been neglected. This can be problematic when there is uncertainty in the number of sparse 
components such as signal sparsity in dynamic systems that can change over time. We present 
a new theoretical framework that handles uncertainty in signal recovery from the perspective 
of recovery success and quality. We show that the signal recovery success in our model is 
more accurate than the success probability analysis in the CS framework. Our model is then 
extended to the case where the success or failure of signal recovery can be relaxed. We represent 
the number of components included in signal recovery with a right-tailed distribution and focus 
on recovery quality. Experimental results confirm the accuracy of our model in dynamic systems. 

Keywords: Compressive sensing, random sampling, dynamic signal sparsity, sparse signal recovery. 


1 Introduction 

Continuous flows of big data are generated by many sources nowadays. Among these, resource 
limited devices occupy a significant portion. For these devices, sensing and transmitting massive 
data are important challenges, as they are concerned with saving resources. 

Compressive sensing (CS) [Il[2l|3llU|5l[6l[7] is a well suited choice for resource limited devices 
because it enables the sensing and compression of massive data without the complexity burden 
imposed by conventional schemes. Recent advances in CS reduce the complexity burden even 
further with random sampling, by which CS schemes have been successfully applied to broader 
application areas lEiEiiin]. 

CS reconstructs the exact signals from the compressed measurements with overwhelming proba¬ 
bility when incoming data can be sparsely represented (i.e., small numbers of components). There¬ 
fore, most CS frameworks are built based on the assumption that incoming data with sparse 
representation can be exactly recovered from an enough number of measurements. 

However, this assumption does not hold in practice when there is no guarantee of enough 
measurements for varying signal sparsity. This uncertainty occurs especially with many dynamic 
systems where the numbers of components change over time. The assumption also implies that 
the reconstruction would fail when input signals have more components (denser) than a predefined 
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threshold. This prevents deriving a tight probabilistic model which exploits the numbers of compo¬ 
nents and measurements in signal recovery. In this regard, recently introduced dynamic CS frame¬ 
works HU DEI [131 El [13 ES] provide the way of reducing the number of necessary measurements 
exploiting temporal correlation between measurements. Nevertheless, a recovery success/quality 
analysis with uncertainty in signal sparsity has not been provided by existing CS frameworks yet. 

This paper presents a new theoretical framework for the random sampling in CS that handles 
impreciseness in signal recovery when the number of measurements lacks for varying signal sparsity. 
Our framework incorporates the beta distribution to present the signal recovery success more 
accurately than the success probability analysis in the CS framework. Furthermore, we relax 
the concept of signal recovery success and present the number of components included in the 
signal recovery as a varying quantity, for which we propose right-tailed distribution modeling. We 
believe our new framework will bridge the gap between success and failure of signal recovery in CS 
frameworks. 


2 Compressive Sensing and Random Sampling 

Compressive sensing, or compressed sampling (CS), is an efficient signal processing framework 
which incorporates signal acquisition and compression simultaneously 13 E!- If a signal can be 
represented by only a few (significant) components with or without the help of a sparsifying basis, 
CS allows it to be efficiently acquired with a number of samples that is far fewer than the signal 
dimension and of the same order as the number of components. 

2.1 Compressing While Sensing 

In CS, a signal is projected onto random vectors whose cardinality is far below the dimension of 
the signal. Consider a signal x G is compactly represented with a sparsifying basis having 
just a few components: x = where s G is the vector of transformed coefficients with a few 
significant coefficients. Here, 'I' could be a basis that makes x sparse in a transform domain such 
as the DCT, wavelet transform domains, or even the canonical basis, i.e., the identity matrix I, if 
X is sparse itself without the help of a transform. 

Definition A signal x is called A'-sparse if it is a linear combination of only K<^N basis vectors 
such that where {ni,...,nid C {!,...,V}; is a coefficient in s; and is a 

column of 

In practice, some signals may not be exactly AT-sparse. Rather, they can be closely approxi¬ 
mated with K basis vectors by ignoring many small coefficients close to zero. This type of signal 
is called compressible urn- 

CS projects x onto a random sensing basis $ G as follows {M<N): 

y = $x = (1) 

where $ should have the restrieted isometry property (Rip)Qa conventional approach for $ to 
satisfy RIP is sampling its independent identically distributed (i.i.d.) elements from the Gaussian 
or other sub-Gaussian distributions whose moment-generating function is bounded by that of the 
Gaussian (e.g., Rademacher/Bernoulli distribution). 

^The random sensing basis ^ have RIP if (1 — (5)||s||2 < |j^4's||2 < (l-|-(i)||s ||2 for small 5 > 0, and this condition 
applies to all TG-sparse s. 
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The system shown in Q is underdetermined, as the number of equations M is smaller than the 
number of variables N, i.e., there are infinitely many x’s that satisfy y = $x. Nevertheless, this 
system can be solved with overwhelming probability exploiting the fact that s is X-sparse. Here 
M = 0{K\og[N/K)) in the case of Gaussian and sub-Gaussian sensing matrices [T7] . 

2.2 Random Sampling 

Random sampling is a variant of CS which can further reduce the computational complexity to 
a constant time um- The random sampling scheme is based on the fact that it is possible to 
construct $ in Q from a random selection of rows from the identity matrix I, which is equivalent 
to the random sampling of coefficients in x. 

Note that the sparsifying basis 'I' should be incoheren^with I, such as the DOT and wavelet 
transform bases, for the successful recovery of the original signal mu- Unless they are incoherent, 
the measurement vector y G in Q would contain zero entries. Here, the number of required 
measurements M is larger than in the cases of Gaussian and sub-Gaussian matrices, that is, M = 
0{K log N). 

2.3 Recovery of Signal 

A signal recovery algorithm takes measurements y, a random sensing matrix and the sparsifying 
basis The sensing matrix $ and sparsifying basis T' are assumed to be known to a decoder. 
The signal recovery algorithm then recovers s knowing that s is sparse. Once we recover s, the 
original signal x can be recovered through x = 'J's. The recovery algorithm reconstructs s by the 
following linear program: 


argmin||s||^ subject to = y. (2) 

The optimization problem in ([^ is solved by a £i-minimization method (basis pursuit) [l8], greedy 
methods such as orthogonal matching pursuit m. or thresholding-based methods such as iterative 
hard thresholding [20) . Choosing a specific algorithm depends on M, N, and K: recovery success 
rates and speed can only be determined by numerical tests [9]j^ In this paper, we reconstruct signals 
by the basis pursuit. 

Specihcally in the case of random sampling, the solution s* to ([^ obeys 

||s* — s ||2 < Cl ■ ||s — Sxlli (3) 

for some constant Ci > 0, where sk is the vector s with all but the largest K components set to 
0. When an original signal is exactly K-sparse, then s = with M = 0{K log N) measurements, 
which implies that the recovery is exact, i.e., s* = s. 


3 A New Perspective on Recovery Success 

The success of signal reconstruction in compressive sensing (CS) is not deterministic. For instance, 
when we say an exact recovery of a A'-sparse signal is achievable with overwhelming probability, it 
implies there is also the chance of recovery not being exact. 

^The two bases ^ and ^ are incoherent when the rows of $ cannot sparsely represent the columns of ^ and vice 
versa. 

^Note that greedy methods are not always fast. 
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Frame 


Figure 1: Recovery error of audio data for different numbers of measurements M over time. Each 
frame has a signal length N = 512. Colors close to blue represent smaller error, whereas colors 
close to red represent larger error. 


Most existing CS literature assumes a sufficient number of measurements M such that an 
exact recovery is almost always achievable [13 i, which is based on the assumption that the 
sparsity K is already known or does not exceed a certain bound. However, the signal sparsity 
in dynamic systems may change over time and an excessive number of measurements may waste 
resources such as network bandwidth and storage space. For example, fig. shows recovery error 
over time for audio data (a 7 second recording of a trumpet solo) [21] , where varying signal sparsity 
incurs different recovery error with a fixed number of measurements over time. Here we cannot 
simply increase the number of measurements to eliminate error, as it is unreasonable in terms of 
compression. Therefore, we propose a new theoretical framework for the random sampling of CS 
and provide a new perspective on signal recovery. 

3.1 Compressive Sensing Framework 

In the random sampling of CS, the number of required measurements M = 0(11 log A^) can be 
detailed as follows [9|: 

M > C'-illn(fV)ln(e“^) (4) 

for some constant O > 0, where e G (0,1) denotes the probability of an inexact recovery of the 
R-sparse signal. In particular, the signal recovery succeeds with a probability of at least 1 — e if 
g holdsQ 

We can then express @ with regard to the probability of failure e, which is given by 

P(s*jis|M,JV,A'):=.<exp(-^-^^). (5) 

Thus, the probability of failure (inexact recovery) P(s* ^ s | M,N,K) is conditional upon M, N, 
and K. Since we are interested in the dynamic signal sparsity R, we model R as a random variable 
with M and N as fixed quantities. 

“^See Theorem 12.20 |9]. 
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If we denote an arbitrary probability density function (pdf) of K as fKik), we can marginalize 
over k and find the upper bound of failure probability as follows: 

P(s* / s I M, AT) = ^ P(s* ^s\M,N,K)- fK{k) dA: < ^ exp f^ik) dk. (6) 

Therefore, we can state that a signal recovery succeeds with a probability of at least 1—/^ exp(—M/(C'' 
ln{N)K))fK{k)dk, given the distribution of signal sparsity fxik)- 

Depending on the form of fK{k), the upper bound in Q may have an analytic solution. In 
particnlar, this is the case when K follows certain distributions such as the inverse Gaussian 
distribution and the gamma distribution^ 

Remark Assuming fxik) = IG(/i, A), the upper bound of Q is 

^/Aexp(A^ - y^2^^Ml{C • In(jV)) + A/2) 

^2M/{C - IniN)) + X 

where /i and A are the mean and the shape parameter of the inverse Gaussian distribution, respec¬ 
tively. 

Remark Assuming fxik) = Gamma(K, 0), the upper bound of Q is 

2 / M I M 

f{K) \C-\n{N)9) C-ln{N)e 

where n and 6 are the shape parameter and the scale parameter of the gamma distribution, respec¬ 
tively; r(-) is the gamma function; is the modified Bessel funtion of the second kind. 



3.2 Modeling Success and Failure 


Unfortunately, the probability of signal recovery failure e given in ([^ does not hold in practice 
because there is a discrepancy between the failure probabilities in the GS framework and actual 
random sampling, as will be further explained in Section 5.1 Thus we have to model the success 


or failure probability of signal recovery from a new perspective. 


We can model the new pdf of signal recovery success using the mixture of the Dirac delta 
function and the beta distribution, which incorporates both stochastic and deterministic cases. We 
introduce ATmin and Almax to denote the minimum and the maximum signal sparsities which yield 
stochastic probability, as opposed to a deterministic result where signal recovery always succeeds 
or always fails. 


Definition Let P(s*=s|M,A^) ;= 11. The pdf of 11 given K is given b}]^ 

( d^TT 1 ) < d^min 

I k )-— \ Beta(Q:^, /3i^) d^min ^ k ^ dCmax. ■ (9) 

[ d(7r) ATmax < k 

®Since K >0, probability distributions supported on semi-infinite intervals, i.e., (0, oo), are rational choices. 
^Betala/c,/Jrr) here is used to denote the pdf of the beta distribution. 


5 













K=20 - B - K=21 » K=22 K=23 — ^ K=24 • K=25 0 K=26 -V- K=27 K=2a - < - K=29 > K=30 | 



Figure 2: Histograms of success probability for various K^s. Success probability distribution for 
each K was obtained with 300 different random signed spike vectors for N = 512 and M = 100. A 
single success probability for each signed spike vector was calculated with 300 experiments. 


Combining this definition with an arbitrary pdf fK^k) of the dynamic signal sparsity K, we 
can find the success probability distribution marginalized over k as follows: 


/n(vr) = / /n|A-(vr | k)fK{k)<lk 
Jk 

fKjnin ri^max fOO 

= / -l)fK{k)dk+ / Beta{aK,PK) ■ fK{k)dk+ / 5(7r)/x(/c) d/c 


'0 


iKr, 






= 6{Tr-l)FK{K^in) + 6{7r){l-FK{Kra^))+ Betaiax,/3 k) ■ fK{k)dk, (10) 

-^min 


where Fk{-) is the cumulative distribution function (CDF) of K. 

The two Dirac delta function terms in (10) can be interpreted as probability masses. Since 
j-^max geta(a:i^,/3i^) • fKik)dk does not have an analytic solution, we compute the values numeri¬ 
cally. 


As an illustrative example, suppose that we examine the success probability by generating 
many different signed spike (±1) vectors for each signal sparsity and then performing experiments 
for each signed spike vector]^ Fig. shows histograms of success probability for various signal 
sparsities, where i^min = 20 and iFmax = 30. 

The success probability shown in Fig. [^naturally follows the beta distribution with its param¬ 
eters a and (3 depending on signal sparsity, i.e., P(s*=s | M,N,K) ~ Beta{aK, Pk)- The beta 
distribution is well known as the conjugate prior for the Bernoulli and the binomial distributions 
which are ideal for modeling success/failure. If more than 300 experiments had been performed in 
Fig.d the variance of each success probability distribution would have been decreased and each 
distribution would have been more sharply peaked. 


^Detailed settings are explained in Section 


5.1 
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3.3 Modeling Accuracy 


Here, we present the main theoretical contribution: the recovery success model defined in is 


tighter than the lower bound of that in the existing CS framework explained in Section 3.1, when 


the number of measurements is not enough. We show the failure probability in the CS framework 
is incapable of reflecting the actual failure probability of signal recovery. It is not only that 
the inequality P(s* ^ s | M,N,K) < exp(—M/(C' • In(A)A)) cannot provide tight probability of 
failure, but the inequality itself is inaccurate. 

This inaccuracy results from the slowly decaying lower bound of success probability, that is, 
1 — exp(—M/(C • In(N)K)). In fact, we can show this lower bound decays slower than a power-law 
decay by the following lemma. 

Lemma 1 (Slackness of Recovery Success Probability) There exists Kq > 0 such that for 
all K > Kq, the lower bound of recovery success probability in the CS framework (Section \3. 1\ ) is 
greater than the value of a power-law-decay function. 

Proof We need to show the following inequality 



( 11 ) 


holds if A > Kq for some Kq > 0, where a > 0. Adding, subtracting, and taking the power K on 
both sides yields 



( 12 ) 


We now use the binomial approximation on the left-hand side: (I — AT “)^ > 1 — K ■ K “ 
Thus we instead prove the following inequality 



(13) 


holds if A > Kq for some Kq > 0. 

If we assume a > I, then adding, subtracting, and taking the power 1/(1 — a) on both sides of 


(13) yields 



(14) 


Setting ATo = (I — exp(—M/(C • ln(A))))^/*^^ we can argue that for all K > Kq, the lower bound 


of recovery success probability is greater than the value of a power-law-decay function. 


□ 


Corollary 2 In the CS framework (Section 3.1), there is always a chance of succeeding at signal 


recovery however large K is. 


Proof The power-law-decay function AT “ in ([Tl|) slowly converges to zero as A —?■ oo: its value is 


noticeably greater than zero even with large K. As the lower bound of recovery success probability 


is greater than the value of the power-law-decay function for all AT > Kq, we can say there is always 


a chance of recovery success however large AT is. 


□ 


We can now show that our recovery success model provides more accurate success probability 
by the following theorem. 
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Theorem 1 The recovery success model in ^ is tighter than the lower bound of recovery success 
probability given by the CS framework (Section \3.1\ ) with a limited number of measurements. 

Proof The claim of the CS framework in Corollary is in fact implausible because it says we can 
even set K > M and there is still a chance of success. We cannot expect signal recovery with a 
number of measurements M less than K. 

On the contrary, our recovery success model can yield P(s* = s | M, N,K) = 0 with a bounded 
In particular, we can let the mean of Beta{aK, Pk), OK/ictR + I^k), converge to zero with 

0- 

Similarly, we show this mean converges to one (P(s* = s | M,N,K) = 1) with /fmin which is 
not so close to zero, whereas the lower bound of the recovery success probability given by the CS 
framework converges to one only if K is very close to zero. 

We can let aK/{oiK + (dx) converge to one with ^ 0. In contrast, 1 — exp(—M/(C • 

ln{N)K)) — )■ 1 if, and only if, K ^ 0. Since 0 < i^min < Kmax < oo, we can argue that our recovery 
success model can provide tighter recovery success probability. □ 

3.4 Parameter Learning in Dynamic Systems 

When the signal sparsity K changes in dynamic systems, it does not change in an abrupt manner; 
rather, it tends to smoothly change over time MM- One simple way to model this correlation 
between K^s is to utilize the Markov model |14) . Here, each K makes up a state and each state is 
associated with the recovery success probability. This can be best modeled by the hidden Markov 
model, where each state K generates success/failure according to the emission probability. 

In our scenario, signal recovery success is observed in an environment where the signal sparsity 
varies over time. We want to estimate parameters of the hidden Markov model, especially the 
emission probabilities. Since our recovery success model employs the beta distribution as conjugate 
distributions (prior and posterior), we can learn its parameters ax and jdx for each state K. 

Specifically, the decoder can observe signal recovery success/failure and corresponding signal 
sparsity K at each decoding step. Then using these emission and state sequences, it can sequentially 
update the parameters ax and fdx for each state K [25]. In order to prevent over-fitting with 
insufficient observations, it is preferrable to have hyperparameters set according to K's. In Fig. 
we can clearly see the trend of ax and fdx for different K's: ax decreases, whereas fdx increases 
as K grows. (Also see the proof of Theorem [^) 


4 Further Analysis on Recovery Quality 


When a signal of interest is not exactly AT-sparse but compressible, as discussed in Section [24] the 
signal recovery in Section 2.3 can be treated from a different perspective [23|. In particular, the 
inequality Q is considered differently. 


If an original signal is compressible, then the quality of a recovered signal is proportional to 
that of the K most significant pieces of information. We get progressively better results as we 
compute more measurements M, since M = 0{K\ogN) [T7|. Therefore, G also makes 
progress on its quality as M increases 


From this viewpoint, the success or failure of signal recovery no longer exists. Rather, we can 
view the number of components included in the signal recovery as a varying quantity. Specifically, 


®The error bound follows ^ as well if 4" is an orthogonal matrix, which is usually the case. 





if a signal recovery is about to fail with a given K, then K can be lowered to make the recovery 
eventually succeed. Here the number of included components K varies for different recoveries and 


signals, as analogous to the success probability in Section 3^ that can be calculated with different 
recoveries and varies for different signals. 

In this regard, can be utilized to infer varying K^s over different recoveries and signals. Here 
our assumption is that the upper bound in Q is tight such that we solve the following optimization 
problem: 

maxK subject to ||s* — s ||2 < Ci • ||s — (15) 


In (15), Cl has to be determined, where the maximum signal sparsity i^max introduced in Section 3.2 


plays a key role to set the upper limit on how large K can be, since K > ilTmax is not reasonable. 

In particular, we can generate a compressible signal Si £ S such that ||sj||^ = and ||sj ||2 = 
Ci 2 for all i, where S is the set containing many different signals; C^j > 0 and Ci^ > 0 being 
constants. For each Sj, we have a set S* which contains many different recoveries sh. Then Ci can 
be found as follows: 

min ||s*-- Sill , , 

Cl = .. \ ( 16 ) 


where s. 




||Si - s,_ 

denotes the compressible signal Sj with all but the largest iFmax components set to 0. 


Varying iF’s obtained through (15) can be represented by a pdf, which has been empirically 
shown to follow the gamma distribution [23]. We are interested in the shape of this pdf, which is 
shown by the following proposition. 

Proposition 1 The pdf of K, the number of components included in the signal recovery of a 
compressible signal, is skewed to the right, i.e., right tailed. 


Proof Since ||si||^ = Cp, and 




^*112 


= 0^2 for all i, we can conceive the same sequence {s„} of 


elements (absolute values) in Sj for all i. Then we have 


P* 111 


N-K 

n=l 


(17) 


Without loss of generality, we consider the partial sum Yln=^ ™ (H"* 1° 6e an arithmetic series 
which can be represented by a quadratic function in terms of K. We also assume the inequality 
constraint in (15) is the equality constraint such that ||s* — s ||2 = Ci ■ ||s — Si^||^. 

If we take the (partial) inverse function of the quadratic function, we have K ~ Tfmax ~ 

Assuming the distribution of ||s* — s ||2 is symmetric (zero skew- 
2 will be compressed as it becomes large, which in turn 


|s*--Si| 


s — s 


s* - s ||2 - (min„_^j _,„2 
ness), this asymptotic relation says 
makes the pdf of K right tailed. 

A similar claim can be made if we consider the partial sum Yln=^ to be a geometric series, 
where K ^ N — log(||s* — SII 2 ). In this case, the pdf of K is skewed to the right as well. □ 


4.1 Error Analysis in Dynamic Systems 

Since the success or failure of signal recovery does not exist in this framework, we instead inves¬ 
tigate the amount of error occurring during the recovery procedure in an expected value sense. In 
particular, the best AT-term approximation ||s — s^Hi in (|^ is known to be bounded as follows 


|s — Siclli < 


If ’ 


(18) 
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where the constant G can be learned by the power-law decay such that each magnitude of com¬ 
ponents in s, sorted in decreasing order, is upper bounded hy Gfi'^. {i = 1,N is the sorted 
index.) 

Then we can analyze the (.2 error E of signal recovery assuming /^(/c) = Gamma(K, 0), which 
is given by 

E = I^C,- dk = - 1,1), (19) 

where B(-, •) is the beta function [23]. Here the pdf fxik) is employed to represent varying iC’sj^ 
In this framework, there is no longer such an indicator as the timely varying signal sparsity 
K in Section because signals are compressible and their coefficients are already populated with 
small, but non-zero, coefficients. Thus, we may assume the same gamma distribution over time, 
whose parameters k and 9 can then be estimated. 


In order to prevent overfitting to insufficient observations, we introduce the conjugate prior for 
the gamma distribution. It is known that the conjugate prior of the gamma distribution has the 
following form |25l |26] . 


P{k,9 I p,q,r,s) 


1 ^exp(—g/0) 


( 20 ) 


where p, q, r, and s are hyperparameters which are sequentially updated with p' = pk, q' = q + k, 
r' = r + 1, and = s -|- 1, respective!}!^ and the normalizing constant Z is 


Z = 


/•oo p«-lr(sK + l) 

Jo ~TW¥^ 


( 21 ) 


Using (19) and (20), we can marginalize over k and 9 to estimate error E as follows: 


E= E ■ P{k, 9 \ p,q,r, s) d9 dK 
J K J 9 

2CiG /■“ p^-^ f°°exp{-q/9) 


z Jo {K-i)r{KYJo 0-+1 

2CiG /■“ p^-^T{sk) 


d9 dfi; 


■Z" Jo {k- i)r{Kyq^ 

which can be computed numerically. 


dK, 


( 22 ) 


5 Experimental Results 

5.1 Recovery Success 

In Section [^ we discussed the discrepancy between the failure probabilities in the CS framework 
and actual random sampling. In order to show this discrepancy, we artificially generated signed 
spikes ±1 at random locations in proportion to desired sparsities and densified these spikes using 
to perform the random sampling. 

For each signal sparsity iF, the actual failure probability can be calculated for different recovery 
experiments. To this end, we adopted a standard optimization method (basis pursuit) to solve the 

®Note that this pdf is different from the one introduced in Section 

^°Here, p', q', r', and s' are updated posterior hyperparameters; A: is a single observation. 

^^We used DCT as the sparsifying basis ^ throughout experiments. 
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Figure 3: Comparison between actual failure probability and failure probabilities given in ^ with 
varying C’s. Actual failure probability of each signal sparsity K was obtained with 300 experiments 
for N = 512 and M = 100. 


optimization problem in Q |27) . Specifically, the primal-dual algorithm based on the interior point 
method was employed to solve Q [T8] . 

Fig. [I shows that the actual failure probability of signal recovery with varying signal sparsity 
does not follow the failure probability given in the CS framework. The failure probability in Q 
cannot model the actual failure probability of signal recovery, regardless of the value chosen for 
constant C. This result confirms Lemma and Corollary 


Moreover, in Section 3.2 we modeled the new pdf of signal recovery success fu{'^) in (10). We 
compared this new pdf with the upper bound of failure in ([^, given a dynamic signal sparsity K. 
Specifically, we employed the inverse Gaussian distribution such that fxik) = IG(30, 200). Fig. 
exhibits the efficacy of our recovery success model, where the lower bounds of success probability 
given in the CS framework fail to capture actual success probability in random sampling case. This 
result confirms Theorem [TJ 


Note that our recovery success model provides the baseline of recovery success for any CS frame¬ 
works that are specifically designed to handle varying signal sparsity. For instance. Fig. shows 
histograms of success probability for various signal sparsities using Modified-CS mn Compared 
with Fig. the success probability shown in Fig. also follows the beta distribution; but success 
probability is higher than that of basis pnrsuit for a given sparsity K (iLmin = 21 and ATmax = 31), 
thanks to the ability of Modified-CS to handle dynamic signal sparsity. The recovery snccess model 
in (§ is still effective here for a theoretical framework, or the recovery success model using basis 
pursuit may promise a minimum guarantee for the recovery success of other CS frameworks. 


We also employed real-world environmental data sets obtained from wireless sensor network 
deployments [2H]: humidity and temperature. In addition, audio data shown in Fig. [^was used 
for comparison as well. Random numbers representing the dynamic signal sparsity K were drawn 
from the inverse Gaussian distribution (/j^(A:) = IG(30,200)) and we used this K to randomly 
choose components sorted in decreasing order; other components were set to zero. Fig. [^displays 
the success probability of signal recovery follows the shape of Fig. 

^^Results were obtained with two frames where the second frame has one more spike than the first frame so that 
Modified-CS could exploit smoothly varying signal sparsity. Histograms in Fig. are the success probability of the 
second frame. 
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Figure 4: Comparison between our new success probability distribution in (10) and the lower 
bounds of success probability obtained by ([^ with varying C’s. The inverse Gaussian distribution 
was used for fK{k). Two probability masses are shown by vertical arrows, where solid boxes atop 
the arrows denote their probabilities. Three vertical dashed/dotted lines represent the lower bounds 
by ([^: C = 0.5 at 0.6781; C = 1 at 0.4450; and C = 2 at 0.2596. Here, TCmin = 20 and it'max = 30. 



Success probability 

Figure 5: Histograms of success probability for various K's using Modified-CS m- Success prob¬ 
ability distribution for each K was obtained with 100 different random signed spike vectors for 
N = 512 and M = 100. A single success probability for each signed spike vector was calculated 
with 100 experiments. Compared with the results with basis pursuit in Fig. success probability 
is higher for a given sparsity K. Note also that 100 experiments resulted in higher variance for 
each K. 


5.2 Recovery Quality 

When a signal is compressible and not exactly IT-sparse, this signal is basically dense. In Section]^ 
we regarded the number of components included in the signal recovery as a varying quantity. We 
are interested in the general shape of this quantity in distribution. In order to verify Proposition!^ 
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(a) humidity (b) temperature (c) audio 


Figure 6: Histograms of success probability for (a) humidity data, (b) temperature data, and (c) 
audio data. Histogram was obtained with 1,500 random number generations (the inverse Gaussian 
distribution) to choose different signals and 100 different experiments for each signal with N = 512 
and M = 100. Note that all histograms closely follow the shape of Fig. 



(a) humidity 




(c) audio 


Figure 7: Distributions of K fitted with gamma distributions for (a) humidity data 
(Gamma(5.69,2.45)), (b) temperature data (Gamma(5.56,2.54)), and (c) audio data 
(Gamma(6.92, 2.21)). Histograms were obtained with 34 different signals and 1,000 different exper¬ 
iments for each signal (a and b); with 153 different signals and 500 different experiments for each 
signal (c), with N = 512 and M = 100. 


we performed experiments using real data sets as well as artificially generated random signed spikes. 

We first provide results with real-world data sets to verify Proposition [T} Fig. displays the 
histograms of 77, the number of components included in each signal recovery, which was obtained 
using the method explained in Section We can identify that Proposition actually holds here, 
as all distributions are skewed to the right. Furthermore, the distributions follow the gamma 
distribution, which is also natural since the gamma distribution has positive skewness, i.e., right 
tailed. 

In addition, random signed spikes were artificially generated in different magnitudes at random 
locations and densified to perform random sampling. In particular, we considered an arithmetic 
sequence of length 50 (2,4,6,..., 98,100), whose elements were placed at random locations in each 
vector. These signals are dense enough to be used for experiments because signal recovery always 
fails when 77 > 30 in our case, as shown in Fig. i Fig-i displays the histogram of 77 and the 
gamma distribution fitting, where we can again see that Proposition!^ holds. 
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Figure 8: Distribution of K fitted with a gamma distribution Gamma(242.81, 0.09), using the 
maximum likelihood estimation. Histogram was obtained with 300 different signals and 300 different 
experiments for each signal, with N = 512 and M = 100. 


Furthermore, we analyze the ^2 error E of signal recovery assuming fxik) = Gamma(K, 0) 
using (19). In order to show its efficacy, we compared the solutions of (19) with real data sets. 
For humidity data, E = 94.3533 while the average £2 norm of data is 564.8585; for temperature 
data, E = 75.5441 while the average £2 norm of data is 627.8038; and for audio data, E = 5.0979 
while the average £2 norm of data is 1.5866. Apart from the case of audio data, (19) provides 
useful estimators for the upper bound of amount of error during recovery. It should be noted that 
this bound is rather loose due to a large constant G in (18), which could be improved with a less 
conservative G. 


6 Conclusion 

We have presented a new theoretical CS framework in random sampling which handles uncertainty 
in signal recovery from a new perspective. The success probability of signal recovery in random 
sampling was investigated when the signal sparsity can vary with an insufficient number of measure¬ 
ments. The success probability analysis in the existing CS framework was shown to be incapable of 
reflecting actual success probability by both theoretical analysis and experiments. On the contrary, 
our recovery success model could closely reflect actual success probability. 

We also considered signals which cannot be exactly represented with sparse representations, 
where we could alternatively view the number of components included in the signal recovery as a 
varying quantity. This quantity was shown by both theoretical analysis and experiments to follow 
a right-tailed distribution such as the gamma distribution. We provided an error analysis for these 
signals. 
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